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Abstract: We give an example of a transient reversible Markov chain that 
almost surely has only a finite number of cutpoints. We explain how this is rele- 
vant to a conjecture of Diaconis and Freedman and a question of Kaimanovich. 
We also answer Kaimanovich's question when the Markov chain is a nearest- 
neighbor random walk on a tree. 

1. Introduction 

While studying extensions of De Finetti's theorem to Markov chains, Diaconis and 
Freedman [3] stated a general conjecture for transient Markov chains {S n }. We 
give a result on cutpoints that is relevant to their conjecture. We begin with some 
background. 

We say that an event A in the space of trajectories of the Markov chain is 
exchangeable if it is invariant under finite permutations, i.e., if (So, Si, . . .) <E A, 
then so is (S v m), . . . , S^tu)) S n +i, ■ ■ ■) for any n and any permutation ir of {0, . . . , n}. 
The cr-field of exchangeable events, £, is called the exchangeable u-field. Let £ be 
the completion of £. A transient process visits each state only finitely often, and so 
for each state x in the state space X there is a random variable V(x) that counts 
the number of visits, V(x) := #{n > 0; S n = x}. We call the collection V := 
{V(x)}xex the occupation numbers of the process. Clearly, V is ^-measurable. A 
natural question, posed by Kaimanovich [6], is to determine under what conditions 
the exchangeable a- field is generated by V. This was motivated by similar issues 
arising in the study [7] of random walks on lamplighter groups. 

Write V n (x) := G [0,n] ; S k = x}. Note that an event A G a(Sj ; j > 0) is 
invariant under permutations of So, ■ • • , S n if and only if A G cr(V n , S n +i, S n +2, ■ ■ ■)■ 
Therefore 

(1) £ = P)cr(V„, S n+ i,S n+2 , ■ ■ ■) • 
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For any Markov chain {S n }, the sequence of transitions {(S n , S n +i)} is also a 
Markov chain; for such chains of transitions, Kaimanovich's question was posed 
earlier as a conjecture by Diaconis and Freedman in [3], To be precise, let M n {x, y) 
be the number of transitions made from x to y up to time n, so that M n (x,y) 
increases to a finite limit M(x,y) as n — > oo. They made the following conjecture 



is always generated (up to completion) by M. 

By comparing (2) to (1), we see that (2) is just the exchangeable cr-field for the 
chain of transitions {(S n , <Sn+i)}- 

James and Peres [5] related the questions above to cutpoints of the Markov chain 
trajectory. Call x a cutpoint if for some k, we have Sk = x and the future of the 
chain, {Sk+i, Sk+2, ■ ■ is disjoint from its past {5*0, Si,..., Sk}- Call Sk a strong 
cutpoint if the probability of a transition from Si to Sj is whenever i < k < j. 
In [5], Conjecture 1.1 was proved under the condition 

(3) the Markov chain {S n } has infinitely many cutpoints almost surely. 

We give a brief outline to illustrate the connection; see [5] for more details. Under 
the assumption (3), the portions ipi, tp2> ■ ■ ■ of the space-time path (n, S n ) between 
successive cutpoints are conditionally independent given M, and the intersection 
(2) is contained in the tail cr-field of the {ipj}j>i, which is trivial (given M) by 
Kolmogorov's zero-one law. Conditional triviality of a cr-field given M means that 
the cr-field is generated by M up to completion. 

James and Peres [5] also showed that if {S n } almost surely has infinitely many 
strong cutpoints, then £ is generated by the occupation numbers. Thus, if every 
transient Markov chain had infinitely many strong cutpoints a.s., then Kaimanov- 
ich's question would be resolved. 

In general, one expects that a random walk that is "very transient" will have 
infinitely many strong cutpoints. As shown in [1, 5, 8], transient random walks on 
Cayley graphs have infinitely many strong cutpoints a.s. More precisely, Lawler [8] 
proved (3) for simple random walk on the lattices Z d for d > 4 and his argument 
applies to strong cutpoints and to any Cayley graph with volume growth at least 
polynomial of degree 5. This was extended, using a different argument, to Z 3 in [5]. 
Blachere [1] extended the argument of [5] and showed that simple random walks on 
all transient Cayley graphs of groups have infinitely many strong cutpoints. 

This raises the natural question of whether every transient Markov chain has 
infinitely many cutpoints a.s.; a positive answer would establish the conjecture of 
Diaconis and Freedman. In Section 3 we show, however, that this is not true, even 
for birth- and-death chains. 

2. Exchangeability, transition counts and trees 

In this section, we show that for transient nearest-neighbor walks on trees, the 
exchangeable cr-field is generated by the occupation numbers. This result was es- 
tablished in the thesis [4] of the first author, but was never published; the proof 




(2) 




1 1, 
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here is shorter than in [4], but relies on the same ideas. Note that the example in 
Section 3 is a nearest-neighbor random walk on a special tree (a halnine) such that 
the walk a.s. has finitely many outpoints, so the proof cannot rely on outpoints. 

Consider a transient Markov chain as in the introduction. If V(x) > 0, let U(x) 
be the state visited by the Markov chain immediately after its last visit to x. For 
completeness, define U(x) ;= x when V(x) = 0. Let W denote the completion of a 
cr-field. 

Theorem 2.1. Let {S n } be a transient Markov chain starting at a fixed state, xq- 
Then £ C a({M (x, y), U{x) ; x, y G X}) . 

Proof. As in Wilson [9], we imagine running the Markov chain by using infinite 
stacks under each of the states. The stack under a state x consists of possible 
successors to x and is generated independently of all other stacks by using the 
transition probabilities from x repeatedly for independent successors. Once the 
stacks are generated, the chain moves by moving to the state given at the top 
of the stack under xq and removing ( "popping" ) the top state under Xq . This is 
repeated from the current state, and so on. The number of states under x that are 
eventually popped equals V(x) and the last one is U(x). Let W(x) be the ordered 
list of states under x that are popped, excluding the last one. Write [W(x)] for 
multi-set of states in W(x), i.e., the unordered list of states (with repetition) in 
W(x). Note that a(M{x,y),U(x) ; x,y E X) = a([W(x)],U(x) ; x eX). 

We first claim that if W(x) is re-ordered for x in some finite set of states A, then 
the resulting chain {S' n } starting at Xo will have the same counts M(x, y) and same 
final exits U (x) . It suffices to prove this when A is a singleton. Moreover, if A is not 
Xq, then we may simply begin the chain when it first reaches A and pop the states 
that are used before then, reducing the situation to A = {.To}. Thus, let A = {xq}. 
The transitions of the chain (So, Si, ... ) describe an Eulerian circuit of a directed 
multi-graph, G. That is, G consists of directed edges (Sk,St+i) connecting vertices 
{Sk} and each vertex has the same number of edges leading to it as leading away 
from it, except that xq has one more edge leading away. When W(xq) is re-ordered, 
the sequence (S' Q , S[, . . . ) does not leave G (while using each edge at most once) 
since the number of possible arrivals to a vertex via an edge of G is at most the 
number of possible departures. Thus, (S' , S[ , . . . ) traverses a subgraph G' of G. If 
we re-order again to the original order, then this argument shows that the resulting 
graph covered, G, is a subgraph of G' . Thus, G' = G. Therefore, the final transition 
counts are the same, as claimed. In addition, the stacks were popped in the same 
order at all vertices other than xq, so their final exits are unchanged, as is U(xq). 

We next claim that the distribution of {S n } given |W(x)] and U(x) for all x G X 
can be represented as follows: Choose randomly and uniformly an ordering W(x) 
for each |W(a;)], independently for each x G X. Then the resulting walk starting 
from xq and determined by these stacks has the same law as the Markov chain. 
To see this, consider the set B of trajectories that correspond to a given collection 
of [W(a;)] and U(x). Let {S n } G B be one such trajectory. Since re-ordering any 
finite set of the corresponding W(x) gives a finite permutation of {S n } with the 
same counts and final exits, B and the conditional Markov chain measure on B 
are preserved. Therefore the Markov chain measure is preserved under re-ordering 
every W(x). The only such invariant measure is the one described, so the claim is 
proved. 

Finally, let C G £. Let B be the set of trajectories that correspond to a given 
collection of [W(x)] and U(x). Since both C and B are invariant under re-ordering 
any finite W(x), so is C H B. In addition, the ordcrings W(x) are independent 
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given all [VK(x)] (and U(x)), so the conditional probability of C given B is or 1 
by Kolmogorov's 0-1 law. Let Do be the union of those B for which the conditional 
probability of C given B is and D\ be the union of the other B. Then P[CnDo] = 
0, so P[CAL>i] = 0. Since D x E a(M(x,y),U(x) ; x,y E X), the theorem is 
proved. □ 

Corollary 2.1. For a transient nearest-neighbor random walk on a tree (with ar- 
bitrary transition probabilities), we have £ = cr(V). 

Proof. Since a transient random walk on a tree T must tend to some end of T, it 
follows that the pointers U{x) are determined by the occupation field V. In view 
of the preceding theorem, it suffices to show that the transition numbers M(x, y) 
are also determined by V. Write Lo = So = xo, and for j > 1 define Lj = U(Lj-i). 
The sequence L = {Lj; j > 0} is known as the loop-erasure of the trajectory 
{Sk ; k > 0}. Consider the finite tree Tp = Tp(Lk) that is spanned by Lk and all 
vertices x with V(x) > and that can be reached from xo without visiting Lk- The 
proof will now follow from the following claim: Given a finite walk from Xq to y on 
a finite tree Tp, the edge transition numbers Mp of the walk are determined by the 
occupation numbers Vp of all vertices except y. The claim is proved by induction 
on the number N of vertices in Tp. The base case N < 2 is clear. For N > 2, 
the tree Tp has some leaf z that is different from y. Let z* denote the neighbor of 
z. Clearly M(z,z*) = V(z) and M(z*,z) = V(z) — \ z=Xo - Removing z from the 
tree and subtracting Vp(z) from Vf(z*) reduces the problem to a tree with N — 1 
vertices and completes the induction step. To apply the claim to our situation, take 
y = Lk and observe that for all vertices w E Tp(Lk) except possibly itself, the 
occupation number V(w) determined by the infinite random walk path coincides 
with Vp(w), the occupation number determined by the portion of that path in 
Tp(Lk). (It is certainly possible that V(Lk) > Vp(Lk), due to excursions of the 
random walk from Lk to the complement of Tp.) □ 



3. A transient birth-and-death chain with finitely many cutpoints 

We shall exhibit a birth-and-death chain, i.e., a nearest-neighbor random walk on 
N, which is transient but has only finitely many cutpoints a.s. We shall use the 
following basic fact about random walks and electrical networks. Let > be given 
for k > 1. (Interpret as the resistance of the edge between k and k+ 1.) Consider 
the birth-and-death chain on {1, 2, . . . , n} where the transition probability from 1 
to 2 is 1, and for k > 1, the transition probability from k to k+ 1 is rjt_i/(r/j_i 
and the transition probability from k to k — 1 is r&/ '(r&_i +r&). Then the probability 
that the chain reaches n before 1 when starting from k equals Y2j=i r jl X^=i r j- 
See [2], §§II.l and IX. 2. Of course, this can also be phrased as a standard gambler's 
ruin calculation. In particular, taking a limit as n — > oo shows that transience is 
equivalent to Y^jLi r j < 00 ■ 

Theorem 3.1. Fix /3 > 1. Let > have the property that X fc _1 (log k)~@ 
for all k > 2, where the symbol x means that the ratio of the two sides is bounded 
above and below by positive constants that do not depend on k. Consider the birth- 
and-death chain onN = {1, 2, . . .} with transition probability r^-i/ (ffc-l + r fe) from 
k to k + 1 and transition probability ru/ '(fk-i + r k) from k to k — 1 for all k > 2. 
(The transition probability from 1 to 2 is 1.) Then this chain is transient and has 
only finitely many cutpoints a.s. 
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Proof. We may assume the chain starts at 1. Since r k < oo, the walk is transient. 

Denote t k := X)j>fe r i- The usual gambler's ruin calculation shows that the 
probability that the walk will have k as a cutpoint is pk =rk/tk- 

Let j < k. Given that k is a cutpoint, let Qk{j) be the conditional probability 
that j is a cutpoint. Then Qk(j) is the probability that a walk starting at j + 1 
visits k + 1 before visiting j, i.e., 

(4) Qk(j) 



(tj-tk+i) ' 

This is also the conditional probability 

P[j is a cutpoint | A: is a cutpoint, Pfc+i] , 

where is any event determined by the future of the walk after it reaches k + 1 
for the first time. 

Let Cj.fc be the set of cutpoints in (2 J ', 2 fe ] and A,-^ := |Cj.fc|. Write a m := 
f[A»,m+i > 0] and 



T 

b m :=min{ £ Q*(fc-i); k € (2"\ 2 m+1 ]} . 



On the event that A m>m+ i > 0, let £ m be the largest cutpoint in C TOjTO _|_i. Bound 
below the expected number of cutpoints in (2" i_1 ,2 m+1 ] by conditioning on the 
last cutpoint in (2 m , 2 m+1 ], if there is one: 



(5) Yl Pi = E t A 



m— l,m+lj 



-1 



> a m E[A m _i !Tn+ i | A„ hm+ i > 0] 

> o, * 

Now x (logj)~^ +1 , whence pj x (j^Sj) 1 f° r J 2. Furthermore, we have 
tk-i-t k+ i x ir k x ir fe _, for 1 < i < 2" 1 " 1 and 2 m < k < 2 m+1 . By (4), this means 
that Qk(k — i) > c/i for some constant c > and i, fc in those ranges, which gives 
in turn that b m > c'm for some constant d > 0. On the other hand, the left-hand 
side of (5) is at most c"(loglog2 m+1 - log log 2 m ) < d"/m for some c", d" < oo. It 
follows that a m = 0(l/m 2 ) is summablc, so that there are a.s. only finitely many 
cutpoints by the Borcl-Cantelli lemma. It also follows that with positive probability, 
there are no cutpoints at all. □ 



4. Concluding remarks 

Given a transient Markov chain {Sj} with a fixed starting state, it is easy to see that 
for any n, the event A n that So, S\ . . . , S n are all cutpoints has positive probability. 
Indeed, starting from a trajectory So, Si, S2 ■ ■ ., consider the corresponding loop- 
erased path {Lj} obtained by erasing cycles in the path as they are created. More 
precisely, L = xo and Lj = U{Lj_\) for j > 0, where U(-) is the ultimate successor 
function defined in Section 2. Fix a sequence of vertices (xi, . . . x„) such that the 
event B n = {(Lq, ...,L n ) = (xq,...,x„)} has P(B n ) > 0. If B n holds for the 
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trajectory {S*}, then Xj = Lj = for some random sequence {fcj}, and we 
define a new trajectory {S*} with S* = Lj for j = 0, . . . , n and = Sk n +i 

for i > 0. For this new trajectory xo, ... ,x n are all cutpoints. We conclude that 
P(A n )>P(B n )UUp(x j -ux j )>0. 

We do not know whether every transient Markov chain has an infinite expected 
number of cutpoints. For any birth-and-death chain, this does hold since (in the 
notation of the preceding proof) J2k>m.P k — J2k>m r k/tm — 1 for every m, whence 
the series *^2 k Pk diverges. 

Another natural question that we cannot answer is whether a simple random 
walk on any transient graph of bounded degree must have infinitely many cutpoints 
a.s. 
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